Methods and systems for detecting suspicious or non-suspicious activities involving a mobile device use

ABSTRACT

Systems and methods are disclosed for detecting a suspicious and/or a non-suspicious activity during an electronic transaction performed by a user device. One method comprises identifying, by a monitoring and detection component, a starting check point in the electronic transaction. The monitoring and detection component may then receive contextual data from one or more sensors of the user device. Based on the contextual data and a machine learning model, the monitoring and detection component may determine whether an expected behavior occurred. Entry of user credentials may be enabled in response to determining that the expected behavior occurred, whereas the electronic transaction may be terminated in response to determining that the expected behavior did not occur.

TECHNICAL FIELD

The present disclosure relates to methods and systems for detectingsuspicious or non-suspicious activities involving a mobile device use.More particularly, the present disclosure relates to methods and systemsfor detecting suspicious or non-suspicious activities based oncontextual data captured by sensors of a user device. The presentdisclosure further relates to methods and systems for analyzingcontextual data to identify suspicious or non-suspicious activitiesusing machine learning.

BACKGROUND

Mobile devices such as smartphones and tablets are becoming more capableof various functions based on the development of hardware, software, andaccessories. One such capability is the processing of a transactionbetween a customer and a merchant using a mobile device. Typically, amerchant uses a mobile device and an accessory device to readinformation from a customer's account card and then process thetransaction through a third party authorization entity. A downloadableapplication can turn an ordinary mobile device into a mobile point ofsale or mobile payment acquiring terminal. While mobile terminals enablemore people to conduct electronic payment transactions in a convenientand flexible manner, security concerns involving mobile terminals arerising.

Thus, a need exists for improving security of mobile device usage invarious types of transactions. More particularly, there is a need forimproving security of electronic payment transactions occurring througha mobile terminal.

The background description provided herein is for the purpose ofgenerally presenting the context of the disclosure. Unless otherwiseindicated herein, the materials described in this section are not priorart to the claims in this application and are not admitted to be priorart, or suggestions of the prior art, by inclusion in this section.

SUMMARY OF THE DISCLOSURE

One embodiment provides a computer-implemented method for detecting asuspicious activity and/or a non-suspicious activity during anelectronic transaction performed by a user device, comprising:identifying, by a monitoring and detection component, a starting checkpoint in the electronic transaction; receiving, by the monitoring anddetection component, contextual data from one or more sensors of theuser device; determining, by the monitoring and detection component,whether an expected behavior occurred based on the received contextualdata and a machine learning model; in response to determining that theexpected behavior occurred, enabling, by the monitoring and detectioncomponent, entry of user credentials; and in response to determiningthat the expected behavior did not occur, terminating, by the monitoringand detection component, the electronic transaction.

One embodiment provides a system for detecting a suspicious activityand/or a non-suspicious activity during an electronic transactionperformed by a user device. The system may comprise one or moreprocessors; and a non-transitory computer readable medium storinginstructions which, when executed by the one or more processors, causethe one or more processors to perform a method comprising: identifying,by a monitoring and detection component, a starting check point in theelectronic transaction; receiving, by the monitoring and detectioncomponent, contextual data from one or more sensors of the user device;determining, by the monitoring and detection component, whether anexpected behavior occurred based on the received contextual data and amachine learning model; in response to determining that the expectedbehavior occurred, enabling, by the monitoring and detection component,entry of user credentials; and in response to determining that theexpected behavior did not occur, terminating, by the monitoring anddetection component, the electronic transaction.

One embodiment provides a non-transitory computer readable medium fordetecting a suspicious activity and/or a non-suspicious activity duringan electronic transaction performed by a user device. The non-transitorycomputer readable medium may store instructions that, when executed byone or more processors, cause the one or more processors to perform amethod comprising: identifying, by a monitoring and detection component,a starting check point in the electronic transaction; receiving, by themonitoring and detection component, contextual data from one or moresensors of the user device; determining, by the monitoring and detectioncomponent, whether an expected behavior occurred based on the receivedcontextual data and a machine learning model; in response to determiningthat the expected behavior occurred, enabling, by the monitoring anddetection component, entry of user credentials; and in response todetermining that the expected behavior did not occur, terminating, bythe monitoring and detection component, the electronic transaction.

Additional objects and advantages of the disclosed embodiments will beset forth in part in the description that follows, and in part will beapparent from the description, or may be learned by practice of thedisclosed embodiments. The objects and advantages of the disclosedembodiments will be realized and attained by means of the elements andcombinations particularly pointed out in the appended claims.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory onlyand are not restrictive of the disclosed embodiments, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this specification, illustrate various exemplary embodiments andtogether with the description, serve to explain the principles of thedisclosed embodiments.

FIG. 1 depicts an exemplary system infrastructure of communicativelycoupled user device(s), a host system, and a training system, accordingto one aspect of the present disclosure.

FIG. 2A depicts an exemplary embodiment of a user device and a trainingsystem, according to one aspect of the present disclosure.

FIG. 2B depicts another exemplary embodiment of a user device and atraining system, according to one aspect of the present disclosure.

FIG. 3A is an exemplary use case diagram illustrating interactionsbetween users and a user device during an electronic paymenttransaction.

FIG. 3B is another exemplary use case diagram illustrating interactionsbetween users and a user device during an electronic paymenttransaction.

FIG. 3C is another exemplary use case diagram illustrating interactionsbetween a user and a user device during an electronic paymenttransaction.

FIG. 4 is a flowchart illustrating an exemplary method of aggregatingcontextual data for model building, according to one aspect of thepresent disclosure.

FIG. 5 is a flowchart illustrating an exemplary method of training amachine learning model, according to one aspect of the presentdisclosure.

FIG. 6 is a flowchart illustrating an exemplary method of analyzingcontextual data to determine an occurrence of an expected activity,according to one aspect of the present disclosure.

FIG. 7A is an exemplary use case diagram illustrating interactionsbetween a user and a user device during unlocking of the user device.

FIG. 7B is an exemplary use case diagram illustrating interactionsbetween users and a user device during unlocking of the user device.

FIG. 8 is a flowchart illustrating an exemplary method of analyzingcontextual data to determine an occurrence of a suspicious activity,according to one aspect of the present disclosure.

FIG. 9 illustrates an implementation of a general computer system thatmay execute techniques presented herein.

DETAILED DESCRIPTION OF EMBODIMENTS

The following embodiments describe methods and systems for detectingsuspicious or non-suspicious activities based on contextual datacaptured by sensors of a user device and, more particularly, foranalyzing contextual data to identify suspicious or non-suspiciousactivities using machine learning.

With the wide spread use of mobile devices, many applications have beendeveloped to enable mobile devices to perform various functions andtransactions. For example, a downloadable software application can turnan ordinary mobile device into a mobile point of sale or mobile paymentacquiring terminal (collectively, a “mobile terminal”). Using the mobileterminal, an electronic payment transaction can be initiated by a sellerwho may take a payment electronically from a buyer. For certaintransactions, an entry of user credentials (via a security challenge)may be required upon a buyer “swiping” or “tapping” a payment vehicle atthe mobile terminal. If such a transaction is initiated at a seller'smobile terminal, the security challenge may be presented on the seller'smobile device. Thus, the seller may be required to present or transferthe device to the buyer who may enter his/her user credentials. Forexample, the buyer may be requested to enter a personal identificationnumber (PIN) associated with the payment vehicle.

To make this process more secure, the disclosed embodiment is directedto detecting suspicious or non-suspicious activities during anelectronic transaction performed at a user device. In one embodiment,the methods and systems of the present disclosure may enabledetermination of whether a user device has been passed from one user toanother (i.e., from a seller to a buyer) for an entry of usercredentials. The determination may be based on contextual data collectedfrom various sensors implemented in the user device. Based on the resultof the determination, the entry of the user credentials may be enabledor disabled. The methods and systems of the present disclosure may beapplicable to a variety of contexts, and may not be limited to the usecase scenarios specifically discussed herein.

The subject matter of the present disclosure will now be described morefully hereinafter with reference to the accompanying drawings, whichform a part hereof, and which show, by way of illustration, specificexemplary embodiments. An embodiment or implementation described hereinas “exemplary” is not to be construed as preferred or advantageous, forexample, over other embodiments or implementations; rather, it isintended to reflect or indicate that the embodiment(s) is/are “example”embodiment(s). Subject matter may be embodied in a variety of differentforms and, therefore, covered or claimed subject matter is intended tobe construed as not being limited to any exemplary embodiments set forthherein; exemplary embodiments are provided merely to be illustrative.Likewise, a reasonably broad scope for claimed or covered subject matteris intended. Among other things, for example, subject matter may beembodied as methods, devices, components, or systems. Accordingly,embodiments may, for example, take the form of hardware, software,firmware or any combination thereof (other than software per se). Thefollowing detailed description is, therefore, not intended to be takenin a limiting sense.

Throughout the specification and claims, terms may have nuanced meaningssuggested or implied in context beyond an explicitly stated meaning.Likewise, the phrase “in one embodiment” as used herein does notnecessarily refer to the same embodiment and the phrase “in anotherembodiment” as used herein does not necessarily refer to a differentembodiment. It is intended, for example, that claimed subject matterinclude combinations of exemplary embodiments in whole or in part.

The terminology used below may be interpreted in its broadest reasonablemanner, even though it is being used in conjunction with a detaileddescription of certain specific examples of the present disclosure.Indeed, certain terms may even be emphasized below; however, anyterminology intended to be interpreted in any restricted manner will beovertly and specifically defined as such in this Detailed Descriptionsection.

Referring now to the appended drawings, FIG. 1 shows an exemplary systeminfrastructure of communicatively coupled user device(s), a trainingsystem, and a host system. In general, FIG. 1 depicts user device(s)110, training system 120, and host system 115, all connected via network105. Network 105 may include the Internet, but may also include othernetworks such as a corporate WAN, cellular network, satellite network,or combination thereof, for example. The network 105 may be employed toenable data communications between the various entities illustrated inFIG. 1 (i.e., user device(s) 110, training system 120, and host system115).

User device 110 may comprise a computing system consistent with orsimilar to that depicted in FIG. 9. In one embodiment, user device 110may be a mobile device comprising a computing system consistent with orsimilar to that depicted in FIG. 9. The term “mobile device” used hereinmay refer to a smart phone, a table, a laptop, a smart watch, a wearabledevice, a gaming device, a handheld computer, a portable media player,or any other mobile or portable computing device. User device 110 may beequipped with various sensors such as, for example, global positioningsystem (GPS) sensors, vision sensors (i.e., cameras), audio sensors(i.e., microphones), light sensors, temperature sensors, radio frequencysensors, direction sensors (i.e., magnetic compasses, magnetometers,gyroscopes), and acceleration sensors (i.e., accelerometers). Userdevice 110 may use these sensors to capture contextual data. Thecontextual data may be used to train a machine learning model and todetect suspicious and/or non-suspicious activities involving deviceusage. In some embodiments, user device 110 may be configured toleverage an external card reader peripheral device to become an ad hocPoint of Sale (POS) platform.

Training system 120 may comprise one or more computer servers consistentwith or similar to that depicted in FIG. 9. Training system 120 maycomprise a single server, or a plurality of servers distributed acrossthe network 105. Notably, training system 120 may receive contextualdata from user device(s) 110 and may train a machine learning modelusing the collected contextual data. Training system 120 may receivecontextual data from one or more user devices 110 in order to analyzeuse patterns across an entire or a portion of a user base. Trainingsystem 120 may provide the trained machine learning model to userdevice(s) 110, in order for the user device(s) 110 to detect suspiciousand/or non-suspicious activities based on contextual data captured inreal-time.

Host system 115 may comprise one or more computer servers consistentwith or similar to that depicted in FIG. 9. Host system 115 may comprisea single server, or a plurality of servers distributed across thenetwork 105. Notably, host system 115 may store software applicationswhich may be requested and downloaded by the user device(s) 110. As willbe described in greater detail below, user device(s) 110 may include auser application 205 installed thereon, to perform electronictransactions, monitor device usage, and detect suspicious and/ornon-suspicious activities. Such an application may be provided from thehost system 115 for download. For example, host system 115 may be aserver hosted by a banking institution, and the software applicationavailable for download at the host system 115 may be an electronicpayment transaction application. As another example, host system 115 maybe a server hosted by an application developer, and may have a varietyof software applications available for download. As another example,host system 115 may be implemented as part of a payment networkcomprising, for example, an issuer computing system, an acquirercomputing system, and a POS system. For instance, host system 115 may beimplemented as part of the issuer computing system or the acquirercomputing system, and may store and provide electronictransaction-related software applications to the user device(s) 110.Notwithstanding the specific exampled discussed above, in general, hostsystem 115 may be any server suitable for storing and providing asoftware application to the user device(s) 110.

FIG. 2A depicts an exemplary embodiment of a user device 110 and atraining system 120, according to one aspect of the present disclosure.User device 110 may comprise a user application 205, an input/output(I/O) interface 220, sensor(s) 225, and a communication interface 230.User application 205 may be a software application downloaded from ahost system 115 and installed on the user device 110. In one embodiment,user application 205 may comprise a transaction component 210 and amonitoring and detection component 215. Transaction component 210 may beconfigured to carry out an electronic transaction for which the userapplication 205 is designed. For example, if the user application 205 isan electronic payment transaction application, transaction component 210may configure the user device 110 as an ad hoc POS platform, and mayenable the user device 110 to present graphical user interface elementsto walk a user through a number of steps in an electronic paymenttransaction. As another example, the user application 205 may be adevice security application, and the transaction component 210 mayenable a user to unlock the user device 110 upon receiving certaincontextual data such as, for example, user biometrics (which may alsoinvolve presenting graphical user interface elements to walk a userthrough one or more biometrics collection steps).

Monitoring and detection component 215 may be configured to collectcontextual data using one or more sensors 225 of the user device 110, inorder to detect suspicious and/or non-suspicious activities involvingdevice usage. As alluded to above, the sensor(s) 225 may include globalpositioning system (GPS) sensors, vision sensors (i.e., cameras), audiosensors (i.e., microphones), light sensors, temperature sensors, radiofrequency sensors, direction sensors (i.e., magnetic compasses,magnetometers, gyroscopes), and acceleration sensors (i.e.,accelerometers). Monitoring and detection component 215 may identify acheck point (i.e., a starting check point) at which to begin capturingcontextual data using the sensor(s) 225, and may also identify anothercheck point (i.e., an ending check point) at which to halt capturingcontextual data using the sensor(s) 225, based on data provided by thetransaction component 210. Monitoring and detection component 215 maytransmit the contextual data to training system 120 for furtherprocessing/analysis, using communication interface 230. I/O interface220 may provide an interface to connect with peripheral devices such as,for example, display devices, integrated input mechanisms (e.g.,keyboards, touch screens, mice, etc.), printers, storage devices,payment input mechanisms (e.g., magnetic card reader devices, smart cardreaders, optical readers, etc.), speakers and headphones, etc.

Training system 120 may comprise a model building component 235 and acommunication interface 240. Model building component 235 may receivecontextual data from the user device 110, and may train a machinelearning model using the contextual data. In one embodiment, modelbuilding component 235 may receive contextual data from a plurality ofuser devices 110, train a machine learning model to be representative ofbehaviors across an entire or a portion of a user base, and provide thetrained machine learning model to the plurality of user devices 110(i.e. to the monitoring and detection components 215 of the userdevice(s) 110). In another embodiment, model building component 235 mayreceive contextual data from a single user device 110, train a machinelearning model to be representative of behaviors localized to that userdevice 110, and provide the trained machine learning model to that userdevice 110 (i.e., to the monitoring and detection component 215 of thatuser device 110).

In yet another embodiment, as shown in FIG. 2B, user device 110 maycomprise a user application 205 including a local model buildingcomponent 245, which may receive contextual data of the user device 110,train a machine learning model to be representative of behaviorsspecific to the user device 110, and provide the trained machinelearning model to the monitoring and detection component 215 of the userdevice 110. Meanwhile, the model building component 235 of the trainingsystem 120 may receive contextual data from a plurality of user devices110, train a machine learning model to be representative of behaviorsacross an entire or a portion of a user base, and may provide thetrained machine learning model to the plurality of user devices 110. Themachine learning model may be continuously or periodically updated asmore contextual data become available at the user devices 110, and theupdated machine learning model may be periodically provided to the userdevices 110.

It should be noted that, although user device 110 and training system120 are shown as separate entities remote from each other in FIGS.2A-2B, the training system 120 may be implemented in the user device 110and the processes performed by the user application 205 (e.g.,transaction component 210 and monitoring and detection component 215)and the model building component 235 may all be performed within theuser device 110. Furthermore, although transaction component 210,monitoring and detection component 215, and local model buildingcomponent 245 are shown as separate components (i.e., applicationlogics) within the user application 205, these components may actuallybe a single component (i.e., an application logic) performing all of thefunctions described below with respect to the transaction component 210,monitoring and detection component 215, and local model buildingcomponent 245, and such a single component may be referred to as thetransaction component 210 or the monitoring and detection component 215.As such, it should be appreciated that the configurations specificallydiscussed herein regarding the arrangement and/or the division of thecomponents depicted in FIGS. 2A-2B are merely exemplary, and differentcombinations of components may be implemented on a single device (e.g.,user device 110) or multiple computing devices (e.g., user device 110and training system 120) to perform the steps described in the presentdisclosure.

As alluded to above, user device 110 may include a user application 205configured to perform an electronic payment transaction (i.e., anelectronic payment transaction application). An electronic paymenttransaction application may enable a seller of goods/services to use theuser device 110 as POS terminal. A user device 110 being used as a POSterminal may be referred to as a “mobile terminal” throughout thepresent disclosure. For example, as shown in FIG. 3A (steps 310 a, 315a, 320 a, and 325 a), users of the user device 110, such as a seller(i.e., user A) and/or a buyer (i.e., user B), may be presented with aseries of graphical user interface screens to complete an electronicpayment transaction. As alluded to above in reference to FIGS. 2A-2B,transaction component 210 of the user application 205 may enable theuser device 110 to present the graphical user interface screens, and mayalso enable transmission of electronic payment transaction-related datato other entities in the payment network (e.g., an acquirer computingsystem, an issuer computing system, etc.) for payment authorization.Further, during the electronic payment transaction, monitoring anddetection component 215 may direct one or more sensors of the userdevice 110 to capture contextual data. For example, as shown in FIG. 3A(images 310 b, 315 b, 320 b, and 325 b), an image sensor (i.e., acamera) may capture facial images of one or more users of the userdevice 110 during the electronic payment transaction, and may analyzethe captured facial images (e.g., face recognition, comparison of therecognized faces, etc.) to detect a suspicious and/or non-suspiciousactivity.

Now with reference to FIG. 3A, more detailed description of thegraphical user interface screens of the user application 205 (i.e.,electronic payment transaction application) and the contextual datacaptured at different stages of the electronic payment transaction willbe provided. Notably, FIG. 3A illustrates a context in which theembodiments contemplated by the present disclosure may be applied. Atstep 310 a, transaction component 210 of the user application 205 maypresent a screen where a seller may enter a transaction amount for asale of goods/services. At step 315 a, in response to the sellerentering the transaction amount, transaction component 210 may confirmthat the transaction amount has been successfully entered by displaying“Amount Entered” with a check mark. Although not shown in FIG. 3A, inaddition to confirming the successful entry of the transaction amount,transaction component 210 may also display instructions for a buyer toprovide a payment vehicle. A payment vehicle may be embodied as aphysical payment card (e.g., a traditional payment card such as a creditcard, a debit card, a pre-paid card, a single-use card, etc.) or avirtual payment card (e.g., a digital wallet, etc.). The userapplication 205 may be configured to process both types of paymentvehicle using a built-in or peripheral device such as, for example, amagnetic card reader, a contactless reader, a contactless near fieldcommunication (NFC) reader, etc. At step 320 a, in response to receivingthe payment vehicle, transaction component 210 may confirm that thepayment vehicle has been successfully “swiped” or “tapped” at the mobileterminal.

While the transaction component 210 may perform tasks directly relatedto processing/completing the electronic payment transaction, monitoringand detection component 215 may collect contextual data using sensors ofthe user device 110, to identify suspicious and/or non-suspiciousactivities that might be associated with the electronic paymenttransaction. In one embodiment, monitoring and detection component 215may direct an image sensor of the user device 110 (i.e., camera) tocapture a facial image of the person using the device. This way,monitoring and detection component 215 may be able to capture the facialimage of the person who is using or facing the device screen when thetransaction amount is entered (i.e., facial image 310 b), when thesuccessful entry of the transaction amount is confirmed (i.e., facialimage 315 b), and/or when the successful provision of the paymentvehicle is confirmed (i.e., facial image 320 b). In the case of thescenario depicted in FIG. 3A, the facial images 310 b, 315 b, 320 bcaptured at the electronic payment transaction stages 310 a, 315 a, 320a are of a seller (i.e., user A). The facial images captured during theelectronic payment transaction may be stored locally and/or remotely forfurther analysis.

With continuing reference to FIG. 3A, once the payment vehicle isaccepted by the mobile terminal (step 320 a), at step 325 a, transactioncomponent 210 may present a screen where the owner of the paymentvehicle (i.e., the buyer) may enter the user credentials associated withthe payment vehicle. The user credentials may include, but may not belimited to, a personal identification number (PIN), a passcode, abirthday, an address, a postal/zip code, an answer to a securityquestion, a social security number, etc. In FIG. 3A, a PIN entry screenis shown as an example. Notably, transaction component 210 may presentthe PIN entry screen only if the monitoring and detection component 215determines that the person attempting to enter the PIN is different fromthe seller, whose facial images were captured at the earlier steps(i.e., 310 a, 315 a, and/or 320 a) and stored. Alternatively,transaction component 210 may present the PIN entry screen subsequent tothe successful provision of the payment vehicle, but may disable inputof numerical digits (e.g., by hiding the numeric keypad) until themonitoring and detection component 215 determines that the personattempting to enter the PIN is different from the seller. Therefore,monitoring and detection component 215 may be configured to identifythat the seller (or the person who set up and/or initiated theelectronic payment transaction) has actually presented or transferredthe user device 110 to a buyer to answer a security challenge (i.e.,provide user credentials) for payment vehicle authentication, and ensurethat it is not the seller who is attempting to provide the answer.

In one embodiment, after the payment vehicle is accepted by the mobileterminal at step 320 a, monitoring and detection component 215 maydirect the image sensor of the user device 110 to capture one or morefacial images of the person facing the device screen. If the facerecognized from the captured one or more facial images (e.g., via facerecognition technology) is different from the seller's face recognizedfrom the facial images 310 b, 315 b, 320 b, the monitoring and detectioncomponent 215 may direct the transaction component 210 to present thePIN entry screen.

In FIG. 3A, the face recognized from the facial image 325 b capturedsubsequent to the acceptance of the payment vehicle is different fromthe face recognized from any of the facial images 310 a, 315 a, 320 b.Accordingly, the PIN entry option is displayed as shown in step 325 a.In another embodiment, after the payment vehicle is accepted by themobile terminal at step 320 a, transaction component 210 may present thePIN entry option with the numeric keypad disabled or hidden.Concurrently, monitoring and detection component 215 may direct theimage sensor of the user device 110 to capture one or more facial imagesof the person facing the device screen, and if the face recognized fromthe captured one or more facial images are different from that of thefacial images 310 b, 315 b, 320 b, may direct the transaction component210 to enable or reveal the numeric keypad for PIN entry.

As will be described in greater detail in the following sections,contextual data used to detect suspicious (e.g., non-transfer of thephone, multiple faces recognized from an image captured at the PIN entrystage, etc.) and/or non-suspicious activities (e.g., transfer of thephone, a single face recognized from an image captured at the PIN entrystage, etc.) may not only include data captured by image sensors of theuser device 110, but may also include data captured by other sensors ofthe device 110. In some cases, using various types of contextual datamay provide more reliable and accurate detection results compared tousing one type of contextual data. Therefore, it should be appreciatedthat, while the description pertaining to FIG. 3A above and FIGS. 3B-3Cbelow may specifically discuss the use of an image sensor, other typesof data may also be captured by other sensors to improve accuracy andreliability of the detection results.

FIG. 3B is another exemplary use case diagram illustrating graphicaluser interface screens of the user application 205 (i.e., electronicpayment transaction application) and the contextual data captured atdifferent stages of the electronic payment transaction. FIG. 3B alsoillustrates a context in which the embodiments contemplated by thepresent disclosure may be applied.

At step 330 a, transaction component 210 of the user application 205 maypresent a screen where a seller may enter a transaction amount for asale of goods/services. At step 335 a, in response to the sellerentering the transaction amount, transaction component 210 may confirmthat the transaction amount has been successfully entered by displaying“Amount Entered” with a check mark. In addition to confirming thesuccessful entry of the transaction amount, transaction component 210may also display instructions for a buyer to provide a payment vehicle.At step 340 a, in response to receiving the payment vehicle, transactioncomponent 210 may confirm that the payment vehicle has been successfully“swiped” or “tapped” at the mobile terminal.

As discussed above in reference to FIG. 3A, to identify suspiciousand/or non-suspicious activities that might be associated with theelectronic payment transaction, monitoring and detection component 215may direct an image sensor of the user device 110 to capture the facialimage of the person who is using or facing the device screen when thetransaction amount is entered (facial image 330 b), when the successfulentry of the transaction amount is confirmed (facial image 335 b),and/or when the successful provision of the payment vehicle is confirmed(facial image 340 b). In the case of the scenario depicted in FIG. 3B,the facial images 330 b, 335 b, 340 b captured at the electronic paymenttransaction stages 330 a, 335 a, 340 a are of a seller (i.e., user A).

Once the payment vehicle is accepted by the mobile terminal (step 340a), at step 345 a, transaction component 210 may present a screen wherea buyer may enter a PIN for authentication. Notably, transactioncomponent 210 may enable the PIN entry (e.g., by displaying a numerickeypad) only if the monitoring and detection component 215 determinesthat the person attempting to enter the PIN is different from theseller. More particularly, transaction component 210 may enable the PINentry only if the facial image captured at the PIN entry stage is of asingle person that is different from the seller, which may indicate thatthe device 110 has been transferred from the seller, and that there isonly one person viewing/entering the PIN. For example, in FIG. 3B, theimage 345 b captured at step 345 a include two faces—of a buyer and asecond person. Although the face of the buyer may suggest that the userdevice 110 has been transferred to the buyer, the face of the secondperson may further suggest that a person other than the buyer may bewatching the PIN entry. Accordingly, the PIN entry may remain disableduntil the second person moves out of the shot.

FIG. 3C is another exemplary use case diagram illustrating graphicaluser interface screens of the user application 205 (i.e., electronicpayment transaction application) and the contextual data captured atdifferent stages of the electronic payment transaction. FIG. 3C alsoillustrates a context in which the embodiments contemplated by thepresent disclosure may be applied.

At step 350 a, transaction component 210 of the user application 205 maypresent a screen where a seller may enter a transaction amount for asale of goods/services. At step 355 a, in response to the sellerentering the transaction amount, transaction component 210 may confirmthat the transaction amount has been successfully entered by displaying“Amount Entered” with a check mark. In addition to confirming thesuccessful entry of the transaction amount, transaction component 210may also display instructions to provide a payment vehicle. At step 360a, in response to receiving the payment vehicle, transaction component210 may confirm that the payment vehicle has been successfully “swiped”or “tapped” at the mobile terminal.

As discussed above in reference to FIG. 3A, to identify suspiciousand/or non-suspicious activities that might be associated with theelectronic payment transaction, monitoring and detection component 215may direct an image sensor of the user device 110 to capture the facialimage of the person who is using or facing the device screen when thetransaction amount is entered (facial image 350 b), when the successfulentry of the transaction amount is confirmed (facial image 355 b),and/or when the successful provision of the payment vehicle is confirmed(facial image 360 b). In the case of the scenario depicted in FIG. 3C,the facial images 350 b, 355 b, 360 b captured at the electronic paymenttransaction stages 350 a, 355 a, 360 a are of a seller (i.e., user A).

Once the payment vehicle is accepted by the mobile terminal (step 360a), at step 365 a, transaction component 210 may present a screen wherea buyer may enter a PIN for authentication. Notably, transactioncomponent 210 may enable the PIN entry (e.g., by displaying a numerickeypad) only if the monitoring and detection component 215 determinesthat the person attempting to enter the PIN is different from theseller. In FIG. 3C, the facial image 365 b captured at step 365 a isthat of the same person (i.e., the seller, who set up or initiated theelectronic payment transaction) compared to the previously-capturedfacial images 350 b, 355 b, 360 b, indicating that the user device 110has stayed with the seller (i.e., non-transfer of the user device 110).Accordingly, the PIN entry may remain disabled until a facial image ofthe buyer, and only of the buyer, is captured by the image sensor,indicating that the user device 110 has been transferred to the buyerand that there is no one else watching the PIN entry but the buyer. Insome embodiments, a hash of the biometrics of individuals using thedevice 110 may be stored for each transaction in order to detectinstances of a third party attempting to enter PINs for a number ofdifferent buyers.

Again, although the description provided in relation to the exemplaryuse cases illustrated in FIGS. 3A-3C only discussed the usage of imagedata (e.g., facial images captured by an image sensor) to detectsuspicious and/or non-suspicious activities, various types of contextualdata captured by multiple sensors may be used to more accurately detectsuspicious and/or non-suspicious activities. For example, while the actof transferring the user device 110 (or the lack thereof) from a seller(i.e., user A) to a buyer (i.e., user B) for a PIN entry may be inferredfrom facial images captured at different stages of the electronicpayment transaction, additional contextual data may be captured usingother sensors to provide more accurate and reliable results. As alludedto above in reference to FIG. 1, user device 110 may contain a number ofsensors, which may provide various information about the environmentaround the user device 110 (i.e., contextual data). For example, a userdevice 110 may use the following sensors to capture various types ofcontextual data to detect suspicious and/or non-suspicious activities:

-   -   Microphones: User device 110 may include at least two        microphones. The microphones used on the device 110 may comprise        a telephone microphone and a background noise microphone used        for noise cancellation. Some devices may also have more than one        microphone in order to record in stereo or surround sound. Each        microphone may detect a slightly different sound. As device 110        is moved within an environment, the sounds detected by each        microphone may change. If the device 110 is rotated, the sounds        measured by the microphones on the device 110 may also appear to        rotate. A full 360-degree rotation may, for instance, create an        illusion that the sounds will swap from one microphone to the        other. By measuring the sound patterns picked up by the multiple        microphones, it may be possible to deduce a rotation of the        device 110 (e.g., a rotation that might be involved when a        mobile terminal is presented to a buyer for a security        challenge), a movement of the device 110, and/or a transfer of        the device 110 from one location/person to another, based on the        changes in sound.    -   Radio frequency sensors: A number of radio frequencies may be        picked up by the user device 110. These may include Wi-Fi,        Bluetooth, Cellular signal, FM radio, etc. A change in these        signals can indicate movement. A combination of these signals        may be used to create a radio transmission map, which may be        used to forensically locate user devices 110.    -   Accelerometer, gyroscope & GPS, and magnetometer: A combination        of the accelerometer, gyroscope, and magnetometer can be used to        detect both the orientation and movement of the user device 110.        A built-in GPS may also be used to determine the location of the        device. The accelerometer may measure the vector displacement of        a user device 110 in three dimensions in relation to the X, Y        and Z axis. The Gyroscope may measure the rotation of a user        device 110, measured around the X, Y and Z axis. The        Magnetometer may be a built-in magnetic compass that may be used        alongside the gyroscope to calculate a reference in relation to        the magnetic north. There may also be a gravity sensor that may        be used to identify which way is up. The GPS may be used to        determine the longitude and latitude of the user device 110. In        one embodiment, when the user device 110 is moved from one party        to another, the combination of the data from the accelerometer,        gyroscope, and the magnetometer may be used to represent that        movement. Using machine learning, an artificial intelligence        (AI) system (e.g., a machine learning model) may gradually be        trained to recognize the patterns that relate to a user device        110 being passed from one person to another.    -   Cameras: The front and back-facing cameras on the user device        110 may be used alongside image recognition to determine        movement in different directions. This movement may then be used        to deduce relocation or rotation of the device 110. If an object        in the background is identified as a distinct object, then the        movement of that object in relation of the user device 110 may        be used to determine that the user device 110 is moving. By        measuring a number of objects in the environment, the movement        of the phone within three dimensional space may be calculated.        The autofocus of the camera on the user device 110 may also be        used to measure depth of field. If an object believed to be        static moves in and out of focus, then it may be deduced that        the user device 110 is moving. The cameras may also be used        alongside face recognition to help prevent the PIN or security        challenge answer from being revealed to a third party.

FIG. 4 is a flowchart illustrating an exemplary method 400 ofaggregating contextual data for model building, according to one aspectof the present disclosure. In particular, the steps of method 400 may beperformed by the monitoring and detection component 215. The aggregatedcontextual data may be used to train a machine learning model in orderto identify a suspicious and/or non-suspicious activity. Therefore, thecontextual data aggregated in method 400 may also be referred to as“training data.”

At step 410, monitoring and detection component 215 may identify astarting check point. The starting check point defines a point fromwhich one or more appropriate sensors of the user device 110 may capturecontextual data. For instance, in the case of an electronic paymenttransaction (e.g., FIGS. 3A-3C), transaction component 210 may alert themonitoring and detection component 215 to begin capturing contextualdata at a particular stage of the electronic payment transaction. Forexample, transaction component 210 may alert the monitoring anddetection component 215 to begin capturing contextual data when agraphical user interface screen for transaction amount entry isdisplayed (e.g., step 310 a in FIG. 3A, step 330 a in FIG. 3B, or step350A in FIG. 3C). In such a case, the point at which the graphical userinterface screen for transaction amount entry is displayed may be thestarting check point. However, the starting check point may be set atany point during a transaction.

At step 415, monitoring and detection component 215 may start receivingcontextual data from the one or more appropriate sensors of the userdevice 110. At step 420, monitoring and detection component 215 mayidentify an ending check point. The ending check point defines a pointat which the one or more appropriate sensors of the user device 110 mayhalt capturing contextual data. Notably, the ending check point and thestarting check point may be set such that one or more sensors of theuser device 110 continue to capture the contextual data when the targetactivity (i.e., suspicious and/or non-suspicious activity) takes place.For example, in the case of an electronic payment transaction (e.g.,FIGS. 3A-3C), the starting point and the ending point may be set suchthat the one or more sensors of the user device 110 capture the transfer(or the lack thereof) of the user device 110 from one person to another(e.g., from a seller to a buyer, etc.).

Once the monitoring and detection component 215 identifies the endingcheck point (step 420), at step 425, monitoring and detection component215 may stop receiving contextual data from the one or more appropriatesensors of the user device 110, and may transmit the aggregatedcontextual data to either one of or both the model building component235 of the training system 120 (to analyze behavior patterns across anentire or a portion of a user base) and the local model buildingcomponent 245 of the user application 205 (to analyze behavior patternslocalized to the user device 110). For example, in the case of anelectronic payment transaction (e.g., FIGS. 3A-3C), the ending checkpoint may be when the PIN is successfully entered by a payment vehicleowner. However, the ending check point may be set at any suitable pointduring a transaction.

In another embodiment, monitoring and detection component 215 may starttransmitting the contextual data to the model building component as soonas it starts receiving the contextual data from the one or more sensors(i.e., from the starting check point). In other words, instead ofwaiting until the ending checking point is identified and sendingaggregated contextual data to the model building component, monitoringand detection component 215 may transmit the contextual data as they arebeing captured, during the time period between the starting check pointand the ending check point.

FIG. 5 is a flowchart illustrating an exemplary method 500 of training amachine learning model, according to one aspect of the presentdisclosure. In particular, the steps of method 500 may be performed byeither one of or both the model building component 235 and the localmodel building component 245. For the sake of brevity, in the followingdiscussion, it will be assumed that the steps of method 500 areperformed by the model building component 235. However, it should beappreciated that the steps of method 500 may also be performed by thelocal model building component 245.

At step 510, model building component 235 may receive contextual datafrom the user device 110 (or the monitoring and detection component 215thereof). At step 515, model building component 235 may prepare thereceived contextual data for model training. Data preparation mayinvolve randomizing the ordering of the contextual data, visualizing thecontextual data to identify relevant relationships between differentvariables, identifying any data imbalances, splitting the contextualdata into two parts where one part is for training a model and the otherpart is for validating the trained model, de-duplicating, normalizing,correcting errors in the contextual data, and so on.

Once the contextual data is prepared (step 515), at step 520, modelbuilding component 235 may train a machine learning model using theprepared contextual data. The trained machine learning model couldanalyze contextual data associated with a suspicious activity to detectone or more patterns associated with the suspicious activity, and couldalso analyze contextual data associated with a non-suspicious activityto detect one or more patterns that are associated with thenon-suspicious activity. In some embodiments, training of the machinelearning model may result in a set of model weights, which can then beused to validate the machine learning model and perform detections viathe machine learning model.

At step 525, model building component 235 may validate the trainedmachine learning model based on, for example, the model weightsgenerated at step 520. For example, the machine learning model may bevalidated by analyzing a set of contextual data that are known torepresent a suspicious or a non-suspicious activity. Accordingly, theaccuracy of the machine learning model and model weights may bedetermined. Once the validation step is complete, at step 530, modelbuilding component 235 may store the trained (and validated) machinelearning model in a system memory or storage. The trained machinelearning model may then be transmitted to and used by the monitoring anddetection component 215 of the user device 110 to detect suspiciousand/or non-suspicious activities involving device usage.

FIG. 6 is a flowchart illustrating an exemplary method 600 ofdetermining an occurrence of an expected activity (i.e., anon-suspicious activity) using a trained machine learning model,according to one aspect of the present disclosure. In particular, thesteps of method 600 may be performed by the monitoring and detectioncomponent 215, and may be performed in contexts described above withreference to FIGS. 3A-3C (i.e., electronic payment transaction).However, it should be appreciated that the steps of method 600 may beused in any context to which they may be applicable.

At step 610, monitoring and detection component 215 may identify astarting check point. As alluded to above in reference to step 410 inFIG. 4, the starting check point defines a point from which one or moreappropriate sensors of the user device 110 may capture contextual data.For instance, in the case of an electronic payment transaction (e.g.,FIGS. 3A-3C), transaction component 210 may alert the monitoring anddetection component 215 to begin capturing contextual data at aparticular stage of the electronic payment transaction. For example,transaction component 210 may alert the monitoring and detectioncomponent 215 to begin capturing contextual data when a graphical userinterface screen for transaction amount entry is displayed (e.g., step310 a in FIG. 3A, step 330 a in FIG. 3B, or step 350A in FIG. 3C). Insuch a case, the point at which the graphical user interface screen fortransaction amount entry is displayed may be the starting check point.However, the starting check point may be set at any point of atransaction.

At step 615, monitoring and detection component 215 may start receivingcontextual data from the one or more sensors of the user device 110.Then, at step 620, monitoring and detection component 215 may determinewhether an expected behavior (e.g., transfer of a user device 110 fromone user to another, etc.) has occurred based on the received contextualdata and a trained machine learning model received from the modeltraining component 235 (or from the local model training component 245).

At step 625, if it is determined that an expected behavior occurredwithin a predetermined time, the method may proceed to step 630 wherethe monitoring and detection component 215 may present a PIN entryscreen or enable PIN entry (or entry of any type of user credentials asdiscussed above), or may direct transaction component 210 to present thePIN entry screen or enable PIN entry. At this point (step 630), themonitoring and detection component 215 may also stop receivingcontextual data from the one or more sensors (or may direct the one ormore sensors to stop transmitting contextual data), as the decision toenable the PIN entry has been made. The point at which to stop receivingcontextual data from the one or more sensors may be referred to as anending check point.

On the other hand, at step 625, if it is determined that an expectedbehavior did not occur within a predetermined time, the method mayproceed to step 635 where the monitoring and detection component 215 mayterminate the transaction, or may direct the transaction component 210to terminate the transaction. At this point (step 635), the monitoringand detection component 215 may also stop receiving contextual data fromthe one or more sensors (or may direct the one or more sensors to stoptransmitting contextual data), as the decision to terminate thetransaction has been made.

In another embodiment, at step 635, the monitoring and detectioncomponent 215 may cause the user device 110 to display instructions tohand the user device 110 to a buyer (i.e., a payment vehicle owner). Inyet another embodiment, at step 635, the monitoring and detectioncomponent 215 may send a notification to the owner of the paymentvehicle (e.g., via an email, a text message, an automated phone call,etc.) that a suspicious activity involving the payment vehicle hasoccurred.

In an alternative embodiment, at step 630, the monitoring and detectioncomponent 215 may keep the PIN entry disabled, and may performadditional authentication processes for added security (e.g., processesdescribed in reference to FIGS. 3A-3C). For example, the monitoring anddetection component 215 may continue to capture one or more facialimages of the user facing the user device 110 even after determiningthat the device transfer has occurred. A facial image captured afterdetermining that the device transfer has occurred may be referred to asa post-transfer facial image. The monitoring and detection component 215may analyze the post-transfer facial images to determine whether morethan one user is staring at the screen of the user device 110. If it isdetermined that there are more than one user staring at the screen ofthe user device 110, the monitoring and detection component 215 mayterminate the transaction or postpone the PIN entry until there is onlyperson in a subsequently-captured post-transfer facial image. If it isdetermined that there is just one user staring at the screen of the userdevice 110, the monitoring and detection component 215 may either i)enable the PIN entry or ii) proceed with yet another authenticationprocess. For example, the monitoring and detection component 215 maycompare the one or more post-transfer facial images to the one or morefacial images that are captured prior to determining that the devicetransfer has occurred (i.e., pre-transfer facial images). If it isdetermined that the face recognized in the post-transfer facial imagesis different from the face recognized in the pre-transfer facial images,the monitoring and detection component 215 may present a PIN entryscreen or enable PIN entry.

At this point, the monitoring and detection component 215 may also stopreceiving contextual data from the one or more sensors (or may directthe one or more sensors to stop transmitting contextual data), as thedecision to enable the PIN entry has been made. If it is determined thatthe face recognized in the post-transfer facial images is the same asthe face recognized in the pre-transfer images, the monitoring anddetection component 215 may terminate the transaction, as discussedabove. At this point, the monitoring and detection component 215 mayalso stop receiving contextual data from the one or more sensors (or maydirect the one or more sensors to stop transmitting contextual data), asthe decision to terminate the transaction has been made.

As alluded to above, in one embodiment, user application 205 may be adevice security application. For example, a device security applicationmay enable a user to unlock the user device 110 using a facialrecognition technology. FIG. 7A is an exemplary user case diagramillustrating graphical user interface screens of the user application205 (i.e., device security application that enables locking/unlocking ofthe user device 110) and the contextual data captured at differentstages of the unlock procedure. Similar to FIGS. 3A-3C, FIG. 7A alsoillustrates a context in which the embodiments contemplated by thepresent disclosure may be applied.

Transaction component 210 of the user application 205 may presentgraphical user interface screens to walk a user through one or morefacial recognition steps to unlock the user device 110. At step 710 a,the user device 110 is “locked” and the lock screen is displayed. If auser wishes to “unlock” the user device 110, the user may touch or swipethe lock screen to enter a “unlock” stage (i.e., facial recognitionstage). At step 715 a, in response to a user touching or swiping thelock screen, transaction component 210 may present a screen indicatingthat the facial recognition is being initiated. For example, the screenmay show a polygon-shaped or circular-shaped box through which a facialimage being captured by the user device 110 may be shown. The screen maydirect or steer the user to adjust the position of user device 110relative to the face, such that the entire face is shown through thebox, in order to unlock the user device 110.

While the transaction component 210 may display graphical user interfacescreens to walk a user through the “unlock” stage (i.e., facialrecognition stage), monitoring and detection component 215 may collectcontextual data using sensors of the user device 110, to identifysuspicious and/or non-suspicious activities that might be associatedwith the unlock procedure. In one embodiment, monitoring and detectioncomponent 215 may direct an image sensor of the user device 110 (i.e.,camera) to capture a facial image of the person attempting to unlock thedevice. For example, as shown in FIG. 7A, facial images of the personusing the device 110 or facing the screen of the device 110 (e.g., 710b, 715 b) may be captured at different stages of the unlock procedure.For example, facial image 710 b may be captured at step 710 a (i.e.,when a user looks at the lock screen or swipes/touches the lock screen)and/or facial image 715 b may be captured at step 715 a (i.e., when auser is presented with a screen indicating that facial recognition isbeing initiated/performed). The monitoring and detection component 215may compare the captured facial images with one or more facial images ofthe persons authorized to use the device, which may have been previouslysaved in the user device 110. At step 720 a, in response to themonitoring and detection component 215 determining that the capturedfacial image matches one of the facial images of the authorized persons,the user device 110 may be unlocked and the user may be presented with ahome screen.

During the unlock procedure, monitoring and detection component 215 mayalso direct other sensors of the user device 110 (i.e., sensors otherthan the image sensor capturing facial images) to capture additionalcontextual data during the unlock procedure, to identify any suspiciousactivities which may not be detectable solely by capturing and analyzingfacial images. For example, an unauthorized user may take or steal theuser device 110 from an authorized user, swipe or touch the lock screen,and hold the user device 110 in front of the authorized user's face tounlock the device. Detecting suspicious activities of this type mayrequire more than just capturing the facial images of the persons facingthe device.

FIG. 7B is another exemplary use case diagram illustrating graphicaluser interface screens of the user application 205 (i.e., devicesecurity application that locks/unlocks the user device 110) and thecontextual data captured at different stages of the unlock procedure.FIG. 7B also illustrates a context in which the embodiments contemplatedby the present disclosure may be applied. Notably, FIG. 7B illustrates ascenario where detection of a suspicious activity may require usingmultiple sensors of the user device 110.

At step 725 a, the user device 110 is “locked” and the lock screen isdisplayed. If a user wishes to “unlock” the user device 110, the usermay touch or swipe the lock screen to enter a “unlock” stage (i.e.,facial recognition stage). At step 730 a, in response to a user touchingor swiping the lock screen, transaction component 210 may present ascreen indicating that the facial recognition is being initiated. Forexample, the screen may show a polygon-shaped or circular-shaped boxthrough which a facial image being captured by the user device 110 isshown. The screen may direct or steer the user to adjust the position ofuser device 110 relative to the face, such that the entire face may beshown through the box, in order to unlock the device 110.

As alluded to above, if the person facing the device screen andattempting to unlock the user device 110 is an unauthorized user, theuser device 110 may remain locked because the facial image of the userdoes not match any of the stored facial images of authorized users.However, the unauthorized user may still attempt to unlock the userdevice 110 by holding the user device 110 in front of an authorizeduser's face. For example, at step 735 a, the unauthorized user may holdthe user device 110 in front of an authorized user's face and may“shake” or “wave” the device 110 such that the entire face of theauthorized user may be captured at the designated location (i.e., withinthe box as shown in 735 b). If only facial images were used to detectsuspicious activities, this behavior (i.e., an unauthorized user“shaking” or “waving” the device 110 in front of the authorized user'sface) could go undetected and the device 110 may be unlocked.

Accordingly, a number of sensors may be used for detection. In oneembodiment, the movement pattern of “shaking” or “waving” the device 110in front of an authorized user (i.e., a victim) in conjunction withmultiple facial images captured at different stages of the unlockingoperation may be used to determine that a suspicious activity is takingplace. As discussed above, one or more of global positioning system(GPS) sensors, vision sensors (i.e., cameras), audio sensors (i.e.,microphones), light sensors, temperature sensors, radio frequencysensors, direction sensors (i.e., magnetic compasses, magnetometers,gyroscopes), and acceleration sensors (i.e., accelerometers) in the userdevice 110 may be used to capture contextual data representative of themovement of the device 110 as well as the biometric measurements (e.g.,facial images).

With continuing reference to FIG. 7B, using the contextual datacollected by various sensors of the user device 110, at step 740 a,monitoring and detection component 215 may determine that a suspiciousactivity is taking place and may keep the user device 110 locked. Insome embodiments, the step of keeping the user device 110 locked mayinvolve notifying the transaction component 210 of the fraudulent loginor unlock attempt, such that the transaction component 210 may keep thedevice locked 110 and may display the lock screen.

FIG. 8 is a flowchart illustrating an exemplary method 800 ofdetermining an occurrence of a suspicious activity using a trainedmachine learning model, according to one aspect of the presentdisclosure. In particular, the steps of method 800 may be performed bythe monitoring and detection component 215, and may be performed incontexts described above with reference to FIGS. 7A-7B (i.e., unlockprocedure). However, it should be appreciated that the steps of method800 may be used in any context to which they may be applicable. Further,methods 400 and 500 of aggregating contextual data representing a targetactivity and training a machine learning model based on the aggregatedcontextual data, respectively, may also be used in conjunction withmethod 800, to determine a target activity discussed below withreference to FIG. 8 (i.e., “shaking” or “waving” user device 110 infront of a victim's face).

At step 810, monitoring and detection component 215 may identify astarting check point. As alluded to above in reference to step 410 inFIG. 4, the starting check point defines a point from which one or moreappropriate sensors of the user device 110 may capture contextual data.For instance, in the case of an unlock procedure (e.g., FIGS. 7A-7B),transaction component 210 may alert the monitoring and detectioncomponent 215 to begin capturing contextual data at a particular stageof the unlock procedure. For example, transaction component 210 mayalert the monitoring and detection component 215 to begin capturingcontextual data when a user touches or swipes the lock screen toinitiate the unlock process (e.g., step 710 a in FIG. 7A, or step 725 ain FIG. 7B). In such a case, the point at which the user touches orswipes the lock screen may be the starting check point. However, thestarting check point may be set at any point during the unlockprocedure. At step 815, monitoring and detection component 215 may startreceiving contextual data from the one or more sensors of the userdevice 110. Then, at step 820, monitoring and detection component 215may determine whether a suspicious activity (e.g., a user “shaking” or“waving” of the user device 110 in front of another user) has occurredbased on the received contextual data and a trained machine learningmodel received from the model training component 235 (or from the localmodel training component 245). At step 825, if it is determined that asuspicious activity occurred or a predetermined time has passed sincethe unlock process was initiated, the method may proceed to step 830where the monitoring and detection component 215 may keep the device 110locked and display the lock screen (or directs the transaction component210 to keep the device 110 locked and to display the lock screen), evenif the facial image captured during the unlock stage matches that of anauthorized user. At this point (step 830), the monitoring and detectioncomponent 215 may also stop receiving contextual data from the one ormore sensors (or may direct the one or more sensors to stop transmittingcontextual data), as the decision to keep the device locked has beenmade. On the other hand, at step 825, if it is determined that asuspicious activity did not occur (e.g., no “shaking” or “waiving” ofthe device 110 in front of an authorized user, and the facial imagecaptured during the unlock stage matches that of an authorized user) anda predetermined time has not passed since the unlock process wasinitiated, the method may proceed to step 835 where the monitoring anddetection component 215 may unlock the device 110 and display the homescreen (or directs the transaction component 210 to unlock the device110 and to display the home screen). At this point (step 835), themonitoring and detection component 215 may also stop receivingcontextual data from the one or more sensors (or may direct the one ormore sensors to stop transmitting contextual data), as the decision tounlock the device 110 has been made. In another embodiment, at step 830,in addition to keeping the device 110 locked, the monitoring anddetection component 215 may send a notification to the owner(s) of theuser device 110 (e.g., via an email, a text message, an automated phonecall, etc.) that a fraudulent login/unlock attempt has taken place.

It should be appreciated that the use cases illustrated in FIGS. 3A-3Cand 7A-7B are merely exemplary, and the embodiments contemplated by thepresent disclosure may also be applicable to variations of the use casesspecifically discussed herein. For example, other use cases may include,but may not be limited to:

-   -   Learning the pattern of a mobile QR code ticket being shown to a        ticket inspector on a train.    -   The pattern of movement to present a loyalty QR code to a QR        code reader in a store.    -   The pattern of movement made when tapping a mobile phone onto a        contactless reader for payments or for transit.    -   Combining the movements of two devices to indicate that they        were both being moved together for a synchronized process such        as a phone to phone payment, or near-field data exchange between        two phones.

The embodiments of the present disclosure may be adjusted or modified toencompass varying use case scenarios. For example, with reference toFIGS. 3A-3C, while the buyer's facial image captured at the paymentvehicle provision confirmation step (e.g., steps 320 a, 340 a, 360 a)may be considered non-suspicious in the United States of America, thesame may not be true for electronic payment transactions occurring in adifferent country. For example, certain countries may have laws in placethat require a mobile terminal to be transferred/presented to the buyerwhen the payment vehicle is provided (i.e., “swiped” or “tapped”) by thebuyer. In such a case, the starting check point and the ending checkpoint discussed in reference to FIGS. 4 and 6 may be adjusted in orderto capture the contextual data indicative of the transfer (or the lackthereof), which is supposed to occur at a point after the transactionamount entry (e.g., steps 310 a, 330 a, 350 a) but before the paymentvehicle provision confirmation (e.g., steps 320 a, 340 a, 360 a).

Unless specifically stated otherwise, as apparent from the followingdiscussions, it is appreciated that throughout the specificationdiscussions utilizing terms such as “processing,” “computing,”“calculating,” “determining”, analyzing” or the like, refer to theaction and/or processes of a computer or computing system, or similarelectronic computing device, that manipulate and/or transform datarepresented as physical, such as electronic, quantities into other datasimilarly represented as physical quantities.

In a similar manner, the term “processor” may refer to any device orportion of a device that processes electronic data, e.g., from registersand/or memory to transform that electronic data into other electronicdata that, e.g., may be stored in registers and/or memory. A “computer,”a “computing machine,” a “computing platform,” a “computing device,” ora “server” may include one or more processors.

FIG. 9 illustrates an implementation of a general computer systemdesignated 900. The computer system 900 can include a set ofinstructions that can be executed to cause the computer system 900 toperform any one or more of the methods or computer based functionsdisclosed herein. The computer system 900 may operate as a standalonedevice or may be connected, e.g., using a network, to other computersystems or peripheral devices.

In a networked deployment, the computer system 900 may operate in thecapacity of a server or as a client user computer in a server-clientuser network environment, or as a peer computer system in a peer-to-peer(or distributed) network environment. The computer system 900 can alsobe implemented as or incorporated into various devices, such as apersonal computer (PC), a tablet PC, a set-top box (STB), a personaldigital assistant (PDA), a mobile device, a palmtop computer, a laptopcomputer, a desktop computer, a communications device, a wirelesstelephone, a land-line telephone, a control system, a camera, a scanner,a facsimile machine, a printer, a pager, a personal trusted device, aweb appliance, a network router, switch or bridge, or any other machinecapable of executing a set of instructions (sequential or otherwise)that specify actions to be taken by that machine. In a particularimplementation, the computer system 900 can be implemented usingelectronic devices that provide voice, video, or data communication.Further, while a single computer system 900 is illustrated, the term“system” shall also be taken to include any collection of systems orsub-systems that individually or jointly execute a set, or multiplesets, of instructions to perform one or more computer functions.

As illustrated in FIG. 9, the computer system 900 may include aprocessor 902, e.g., a central processing unit (CPU), a graphicsprocessing unit (GPU), or both. The processor 902 may be a component ina variety of systems. For example, the processor 902 may be part of astandard personal computer or a workstation. The processor 902 may beone or more general processors, digital signal processors, applicationspecific integrated circuits, field programmable gate arrays, servers,networks, digital circuits, analog circuits, combinations thereof, orother now known or later developed devices for analyzing and processingdata. The processor 902 may implement a software program, such as codegenerated manually (i.e., programmed).

The computer system 900 may include a memory 904 that can communicatevia a bus 908. The memory 904 may be a main memory, a static memory, ora dynamic memory. The memory 904 may include, but is not limited tocomputer readable storage media such as various types of volatile andnon-volatile storage media, including but not limited to random accessmemory, read-only memory, programmable read-only memory, electricallyprogrammable read-only memory, electrically erasable read-only memory,flash memory, magnetic tape or disk, optical media and the like. In oneimplementation, the memory 904 includes a cache or random-access memoryfor the processor 902. In alternative implementations, the memory 904 isseparate from the processor 902, such as a cache memory of a processor,the system memory, or other memory. The memory 904 may be an externalstorage device or database for storing data. Examples include a harddrive, compact disc (“CD”), digital video disc (“DVD”), memory card,memory stick, floppy disc, universal serial bus (“USB”) memory device,or any other device operative to store data. The memory 904 is operableto store instructions executable by the processor 902. The functions,acts or tasks illustrated in the figures or described herein may beperformed by the programmed processor 902 executing the instructionsstored in the memory 904. The functions, acts or tasks are independentof the particular type of instructions set, storage media, processor orprocessing strategy and may be performed by software, hardware,integrated circuits, firm-ware, micro-code and the like, operating aloneor in combination. Likewise, processing strategies may includemultiprocessing, multitasking, parallel processing and the like.

As shown, the computer system 900 may further include a display unit910, such as a liquid crystal display (LCD), an organic light emittingdiode (OLED), a flat panel display, a solid-state display, a cathode raytube (CRT), a projector, a printer or other now known or later developeddisplay device for outputting determined information. The display 910may act as an interface for the user to see the functioning of theprocessor 902, or specifically as an interface with the software storedin the memory 904 or in the drive unit 906.

Additionally or alternatively, the computer system 900 may include aninput device 912 configured to allow a user to interact with any of thecomponents of system 900. The input device 912 may be a number pad, akeyboard, or a cursor control device, such as a mouse, or a joystick,touch screen display, remote control, or any other device operative tointeract with the computer system 900.

The computer system 900 may also or alternatively include a disk oroptical drive unit 906. The disk drive unit 906 may include acomputer-readable medium 922 in which one or more sets of instructions924, e.g. software, can be embedded. Further, the instructions 924 mayembody one or more of the methods or logic as described herein. Theinstructions 924 may reside completely or partially within the memory904 and/or within the processor 902 during execution by the computersystem 900. The memory 904 and the processor 902 also may includecomputer-readable media as discussed above.

In some systems, a computer-readable medium 922 includes instructions924 or receives and executes instructions 924 responsive to a propagatedsignal so that a device connected to a network 105 can communicatevoice, video, audio, images, or any other data over the network 105.Further, the instructions 924 may be transmitted or received over thenetwork 105 via a communication port or interface 920, and/or using abus 908. The communication port or interface 920 may be a part of theprocessor 902 or may be a separate component. The communication port 920may be created in software or may be a physical connection in hardware.The communication port 920 may be configured to connect with a network105, external media, the display 910, or any other components in system900, or combinations thereof. The connection with the network 105 may bea physical connection, such as a wired Ethernet connection or may beestablished wirelessly as discussed below. Likewise, the additionalconnections with other components of the system 900 may be physicalconnections or may be established wirelessly. The network 105 mayalternatively be directly connected to the bus 908.

While the computer-readable medium 922 is shown to be a single medium,the term “computer-readable medium” may include a single medium ormultiple media, such as a centralized or distributed database, and/orassociated caches and servers that store one or more sets ofinstructions. The term “computer-readable medium” may also include anymedium that is capable of storing, encoding, or carrying a set ofinstructions for execution by a processor or that cause a computersystem to perform any one or more of the methods or operations disclosedherein. The computer-readable medium 922 may be non-transitory, and maybe tangible.

The computer-readable medium 922 can include a solid-state memory suchas a memory card or other package that houses one or more non-volatileread-only memories. The computer-readable medium 922 can be arandom-access memory or other volatile re-writable memory. Additionallyor alternatively, the computer-readable medium 922 can include amagneto-optical or optical medium, such as a disk or tapes or otherstorage device to capture carrier wave signals such as a signalcommunicated over a transmission medium. A digital file attachment to ane-mail or other self-contained information archive or set of archivesmay be considered a distribution medium that is a tangible storagemedium. Accordingly, the disclosure is considered to include any one ormore of a computer-readable medium or a distribution medium and otherequivalents and successor media, in which data or instructions may bestored.

In an alternative implementation, dedicated hardware implementations,such as application specific integrated circuits, programmable logicarrays and other hardware devices, can be constructed to implement oneor more of the methods described herein. Applications that may includethe apparatus and systems of various implementations can broadly includea variety of electronic and computer systems. One or moreimplementations described herein may implement functions using two ormore specific interconnected hardware modules or devices with relatedcontrol and data signals that can be communicated between and throughthe modules, or as portions of an application-specific integratedcircuit. Accordingly, the present system encompasses software, firmware,and hardware implementations.

The computer system 900 may be connected to one or more networks 10026.The network 105 may define one or more networks including wired orwireless networks. The wireless network may be a cellular telephonenetwork, an 802.11, 802.16, 802.20, or WiMax network. Further, suchnetworks may include a public network, such as the Internet, a privatenetwork, such as an intranet, or combinations thereof, and may utilize avariety of networking protocols now available or later developedincluding, but not limited to TCP/IP based networking protocols. Thenetwork 105 may include wide area networks (WAN), such as the Internet,local area networks (LAN), campus area networks, metropolitan areanetworks, a direct connection such as through a Universal Serial Bus(USB) port, or any other networks that may allow for data communication.The network 105 may be configured to couple one computing device toanother computing device to enable communication of data between thedevices. The network 105 may generally be enabled to employ any form ofmachine-readable media for communicating information from one device toanother. The network 105 may include communication methods by whichinformation may travel between computing devices. The network 105 may bedivided into sub-networks. The sub-networks may allow access to all ofthe other components connected thereto or the sub-networks may restrictaccess between the components. The network 105 may be regarded as apublic or private network connection and may include, for example, avirtual private network or an encryption or other security mechanismemployed over the public Internet, or the like.

In accordance with various implementations of the present disclosure,the methods described herein may be implemented by software programsexecutable by a computer system. Further, in an exemplary, non-limitedimplementation, implementations can include distributed processing,component/object distributed processing, and parallel processing.Alternatively, virtual computer system processing can be constructed toimplement one or more of the methods or functionality as describedherein.

Although the present specification describes components and functionsthat may be implemented in particular implementations with reference toparticular standards and protocols, the disclosure is not limited tosuch standards and protocols. For example, standards for Internet andother packet switched network transmission (e.g., TCP/IP, UDP/IP, HTML,HTTP) represent examples of the state of the art. Such standards areperiodically superseded by faster or more efficient equivalents havingessentially the same functions. Accordingly, replacement standards andprotocols having the same or similar functions as those disclosed hereinare considered equivalents thereof.

It will be understood that the steps of methods discussed are performedin one embodiment by an appropriate processor (or processors) of aprocessing (i.e., computer) system executing instructions(computer-readable code) stored in storage. It will also be understoodthat the invention is not limited to any particular implementation orprogramming technique and that the invention may be implemented usingany appropriate techniques for implementing the functionality describedherein. The invention is not limited to any particular programminglanguage or operating system.

It should be appreciated that in the above description of exemplaryembodiments of the invention, various features of the invention aresometimes grouped together in a single embodiment, figure, ordescription thereof for the purpose of streamlining the disclosure andaiding in the understanding of one or more of the various inventiveaspects. This method of disclosure, however, is not to be interpreted asreflecting an intention that the claimed invention requires morefeatures than are expressly recited in each claim. Rather, as thefollowing claims reflect, inventive aspects lie in less than allfeatures of a single foregoing disclosed embodiment. Thus, the claimsfollowing the Detailed Description are hereby expressly incorporatedinto this Detailed Description, with each claim standing on its own as aseparate embodiment of this invention.

Furthermore, while some embodiments described herein include some butnot other features included in other embodiments, combinations offeatures of different embodiments are meant to be within the scope ofthe invention, and form different embodiments, as would be understood bythose skilled in the art. For example, in the following claims, any ofthe claimed embodiments can be used in any combination.

Furthermore, some of the embodiments are described herein as a method orcombination of elements of a method that can be implemented by aprocessor of a computer system or by other means of carrying out thefunction. Thus, a processor with the necessary instructions for carryingout such a method or element of a method forms a means for carrying outthe method or element of a method. Furthermore, an element describedherein of an apparatus embodiment is an example of a means for carryingout the function performed by the element for the purpose of carryingout the invention.

In the description provided herein, numerous specific details are setforth. However, it is understood that embodiments of the invention maybe practiced without these specific details. In other instances,well-known methods, structures and techniques have not been shown indetail in order not to obscure an understanding of this description.

Similarly, it is to be noticed that the term coupled, when used in theclaims, should not be interpreted as being limited to direct connectionsonly. The terms “coupled” and “connected,” along with their derivatives,may be used. It should be understood that these terms are not intendedas synonyms for each other. Thus, the scope of the expression a device Acoupled to a device B should not be limited to devices or systemswherein an output of device A is directly connected to an input ofdevice B. It means that there exists a path between an output of A andan input of B which may be a path including other devices or means.“Coupled” may mean that two or more elements are either in directphysical or electrical contact, or that two or more elements are not indirect contact with each other but yet still co-operate or interact witheach other.

Thus, while there has been described what are believed to be thepreferred embodiments of the invention, those skilled in the art willrecognize that other and further modifications may be made theretowithout departing from the spirit of the invention, and it is intendedto claim all such changes and modifications as falling within the scopeof the invention. For example, any formulas given above are merelyrepresentative of procedures that may be used. Functionality may beadded or deleted from the block diagrams and operations may beinterchanged among functional blocks. Steps may be added or deleted tomethods described within the scope of the present invention.

The above disclosed subject matter is to be considered illustrative, andnot restrictive, and the appended claims are intended to cover all suchmodifications, enhancements, and other implementations, which fallwithin the true spirit and scope of the present disclosure. Thus, to themaximum extent allowed by law, the scope of the present disclosure is tobe determined by the broadest permissible interpretation of thefollowing claims and their equivalents, and shall not be restricted orlimited by the foregoing detailed description. While variousimplementations of the disclosure have been described, it will beapparent to those of ordinary skill in the art that many moreimplementations and implementations are possible within the scope of thedisclosure. Accordingly, the disclosure is not to be restricted exceptin light of the attached claims and their equivalents.

1-20. (canceled)
 21. A computer-implemented method of detecting asuspicious activity during an electronic transaction, comprising:determining, by a monitoring and detection component, that a transfer ofa user device from a first user to a second user occurred during theelectronic transaction; receiving, by the monitoring and detectioncomponent, facial images of the first user and the second user duringthe transfer of the user device; generating, by the monitoring anddetection component, one or more user interfaces for display by the userdevice prompting entry of user credentials by the first user and thesecond user upon determining the facial images of the first user matchesbiometric data of a registered seller and the facial images of thesecond user matches biometric data of a registered buyer; and inhibitingor postponing, by the monitoring and detection component, the entry ofthe user credentials by the second user upon determining a presence of athird user within a proximity threshold of the user device and thesecond user, wherein the third user is not a registered user.
 22. Thecomputer-implemented method of claim 21, further comprising:configuring, by the monitoring and detection component, to collectcontextual data of one or more users via one or more sensors of aplurality of user devices wherein the contextual data includes thebiometric data and device movement data; preparing, by a model buildingcomponent, the collected contextual data by randomizing an ordering ofthe contextual data, visualizing the contextual data to identifyrelevant relationships between different variables, and identifying dataimbalances; and training, by the model building component, a machinelearning model using the prepared contextual data to detect one or morepatterns associated with the suspicious activity during the electronictransaction.
 23. The computer-implemented method of claim 22, wherein ahash of the biometric data of the one or more users is stored for eachof the electronic transaction to detect the third user attempting toenter the user credentials.
 24. The computer-implemented method of claim22, further comprising: determining, by the monitoring and detectioncomponent, a starting check point to capture the contextual data of theone or more users, wherein the starting check point is initiated upon apresentation of the one or more user interfaces in the user device forthe entry of a transaction amount; and determining, by the monitoringand detection component, an ending check point to halt the capturing ofthe contextual data of the one or more users, wherein the ending checkpoint is activated upon successful entry of the user credentials. 25.The computer-implemented method of claim 24, further comprising:transmitting, by the monitoring and detection component, an aggregatedcontextual data upon determining the ending check point, wherein theaggregated contextual data is transmitted to the model buildingcomponent to analyze behavior patterns across an entire user base and alocal model building component of a user application in the user deviceto analyze the behavior patterns localized to the user device.
 26. Thecomputer-implemented method of claim 21, further comprising:deactivating, by the monitoring and detection component, the entry ofthe user credentials upon determining the facial images of the firstuser matches the facial images of the second user, wherein thedeactivation of the entry of the user credentials include a hiding of anumeric keypad to disable an input of numerical digits; and generating,by the monitoring and detection component, a visual and/or an auralnotification in the user device to transfer the user device to theregistered buyer or on an occurrence of the suspicious activity.
 27. Thecomputer-implemented method of claim 26, further comprising: activating,by the monitoring and detection component, the entry of the usercredentials upon determining the facial images of the first user isdifferent from the facial images of the second user, wherein the facialimages of the second user matches the biometric data of the registeredbuyer, and wherein the activation of the entry of the user credentialsinclude a revealing of the numeric keypad to enable the input of thenumerical digits.
 28. The computer-implemented method of claim 22,wherein the contextual data comprises image data and sound data receivedfrom the one or more sensors of the plurality of user devices, whereinthe sound data is processed to determine a sound pattern to deduce amovement and the transfer of the user device.
 29. Thecomputer-implemented method of claim 22, wherein the contextual datacomprises vector displacement measurements received from anaccelerometer of the plurality of user devices and rotation measurementsreceived from a gyroscope and a magnetometer of the plurality of userdevices.
 30. The computer-implemented method of claim 21, wherein theelectronic transaction is terminated upon determining the transfer ofthe user device from the first user to the second user did not occurwithin a pre-determined time threshold.
 31. A system for detecting asuspicious activity during an electronic transaction, comprising:determining, by a monitoring and detection component, that a transfer ofa user device from a first user to a second user occurred during theelectronic transaction; receiving, by the monitoring and detectioncomponent, facial images of the first user and the second user duringthe transfer of the user device; generating, by the monitoring anddetection component, one or more user interfaces for display by the userdevice prompting entry of user credentials by the first user and thesecond user upon determining the facial images of the first user matchesbiometric data of a registered seller and the facial images of thesecond user matches biometric data of a registered buyer; and inhibitingor postponing, by the monitoring and detection component, the entry ofthe user credentials by the second user upon determining a presence of athird user within a proximity threshold of the user device and thesecond user, wherein the third user is not a registered user.
 32. Thesystem of claim 31, further comprising: configuring, by the monitoringand detection component, to collect contextual data of one or more usersvia one or more sensors of a plurality of user devices wherein thecontextual data includes the biometric data and device movement data;preparing, by a model building component, the collected contextual databy randomizing an ordering of the contextual data, visualizing thecontextual data to identify relevant relationships between differentvariables, and identifying data imbalances; and training, by the modelbuilding component, a machine learning model using the preparedcontextual data to detect one or more patterns associated with thesuspicious activity during the electronic transaction.
 33. The system ofclaim 32, wherein a hash of the biometric data of the one or more usersis stored for each of the electronic transaction to detect the thirduser attempting to enter the user credentials.
 34. The system of claim32, further comprising: determining, by the monitoring and detectioncomponent, a starting check point to capture the contextual data of theone or more users, wherein the starting check point is initiated upon apresentation of the one or more user interfaces in the user device forthe entry of a transaction amount; and determining, by the monitoringand detection component, an ending check point to halt the capturing ofthe contextual data of the one or more users, wherein the ending checkpoint is activated upon successful entry of the user credentials. 35.The system of claim 34, further comprising: transmitting, by themonitoring and detection component, an aggregated contextual data upondetermining the ending check point, wherein the aggregated contextualdata is transmitted to the model building component to analyze behaviorpatterns across an entire user base and a local model building componentof a user application in the user device to analyze the behaviorpatterns localized to the user device.
 36. The system of claim 31,further comprising: deactivating, by the monitoring and detectioncomponent, the entry of the user credentials upon determining the facialimages of the first user matches the facial images of the second user,wherein the deactivation of the entry of the user credentials include ahiding of a numeric keypad to disable an input of numerical digits; andgenerating, by the monitoring and detection component, a visual and/oran aural notification in the user device to transfer the user device tothe registered buyer or on an occurrence of the suspicious activity. 37.The system of claim 36, further comprising: activating, by themonitoring and detection component, the entry of the user credentialsupon determining the facial images of the first user is different fromthe facial images of the second user, wherein the facial images of thesecond user matches the biometric data of the registered buyer, andwherein the activation of the entry of the user credentials include arevealing of the numeric keypad to enable the input of the numericaldigits.
 38. A non-transitory computer readable medium storinginstructions that, when executed by one or more processors, cause theone or more processors to perform a method of detecting a suspiciousactivity during an electronic transaction, the method comprising:determining, by a monitoring and detection component, that a transfer ofa user device from a first user to a second user occurred during theelectronic transaction; receiving, by the monitoring and detectioncomponent, facial images of the first user and the second user duringthe transfer of the user device; generating, by the monitoring anddetection component, one or more user interfaces for display by the userdevice prompting entry of user credentials by the first user and thesecond user upon determining the facial images of the first user matchesbiometric data of a registered seller and the facial images of thesecond user matches biometric data of a registered buyer; and inhibitingor postponing, by the monitoring and detection component, the entry ofthe user credentials by the second user upon determining a presence of athird user within a proximity threshold of the user device and thesecond user, wherein the third user is not a registered user.
 39. Thenon-transitory computer readable medium of claim 38, further comprising:configuring, by the monitoring and detection component, to collectcontextual data of one or more users via one or more sensors of aplurality of user devices wherein the contextual data includes thebiometric data and device movement data; preparing, by a model buildingcomponent, the collected contextual data by randomizing an ordering ofthe contextual data, visualizing the contextual data to identifyrelevant relationships between different variables, and identifying dataimbalances; and training, by the model building component, a machinelearning model using the prepared contextual data to detect one or morepatterns associated with the suspicious activity during the electronictransaction.
 40. The non-transitory computer readable medium of claim39, wherein a hash of the biometric data of the one or more users isstored for each of the electronic transaction to detect the third userattempting to enter the user credentials.